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Abstract 

The paper considers the problem of estimating ap > 2 dimensional 
mean vector of a multivariate conditionally normal distribution under 
quadratic loss. The problem of this type arises when estimating the 
parameters in a continuous time regression model with a non-Gaussian 
Ornstein-Uhlenbeck process driven by the mixture of a Brownian mo- 
tion and a compound Poisson process. We propose a modification of 
the James-Stein procedure of the form 9*{Y) = (1 — c/||y||)y, where 
Y is an observation and c > is a special constant. This estimate al- 
lows one to derive an explicit upper bound for the quadratic risk and 
has a significantly smaller risk than the usual maximum likelihood 
estimator for the dimensions p > 2. This procedure is applied to the 
problem of parametric estimation in a continuous time conditionally 
Gaussian regression model and to that of estimating the mean vector 
of a multivariate normal distribution when the covariance matrix is 
unknown and depends on some nuisance parameters. 
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1 Introduction 



In 1961, James and Stein [7j considering the problem of estimating the mean 
vector ^ of a p-dimensional normal distributed random vector Y with a co- 
variance matrix Ip introduced an estimator 

which outperforms the maximum likelihood estimate (MLE) 

0ml = Y (2) 
for dimension p > 3, under the common quadratic risk 

R{9,9)=Ee\\9-9f, (3) 
in the sense that for all parameter values 9 

R{9,9js)<R{9,9ml). 

This unexpected result draw a great interest of mathematical statisticians 
and stimulated a number of authors to contribute to the theory of improved 
estimation by extending the problem of James and Stein in different direc- 
tions to more general models with unknown covariance matrix and consid- 
ering other types of estimates (see [H [2], [6l [9] for more details and other 
references). A considerable effort has been directed towards the problems of 
improved estimation in non-Gaussian models with the spherically symmetric 
distributions (see |3ii5j) and in the non-parametric regression models |4j. 

Now the James-Stein estimator and other improved shrinkage estimators 
are widely used in econometrics and the problems associated with the signal 
processing. 

In this paper we will consider the problem of estimating the mean in a 
conditionally Gaussian distribution. Suppose that the observation F is a 
p-dimensional random vector which obeys the equation 

Y = 9 + ^, (4) 

where 6' is a constant vector parameter, is a conditionally Gaussian random 
vector with a zero mean and the covariance matrix T>{Q), i.e. Law{^\Q) = 
Mp{0,V{Q)), where Q is some fixed cx-algebra. 
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We propose to consider a shrinkage estimator of the form 

where c is a positive constant which will be specified below. It will be shown 
that such an estimator allows one to obtain an explicit upper bound for 
the quadratic risk in case of the regression model (jlj) with a conditionally 
Gaussian noise. Theorem 12.11 in Section [2] claims that the estimator ([5]) 
outperforms the maximum likelihood estimate 9ml uniformly in 9 from any 
compact set G C for any dimension p starting from two. In Section 
[3l we apply the estimator ([5]) to solve the problem of improved parametric 
estimation in the regression model in continuous time with a non-Gaussian 
noise. 

The rest of the paper is organized as follows. In Section [21 we impose 
some conditions on the random covariance matrix 'D{Q) and derive to upper 
bound for the difference of risks 

A(0) ■.= R{9\9)-R{9ml.9) 

corresponding to 9* and 9ml respectively. In Section HJ the estimate ^ 
is used for the parameter estimation in a discrete time regression with a 
Gaussian noise depending on some nuisance parameters. Appendix contains 
some technical results. 

2 The upper bound for the estimate risk 

In this section we will derive an upper bound for the risk of estimate ([5]) 
under some conditions on the random covariance matrix T>{Q). 
Assume that 

(Ci) There exists a positive constant A*, such that the minimal eigenvalue 
of matrix T>{Q) satisfies the inequality 

(VlQ)) > a.s. 

(C2) The maximal eigenvalue of the matrix T>{Q) is bounded on some 
compact set O C from above, i.e. 

sup EgAmax 

(VlQ)) < a*, 

eee 
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(5) 



where a* is some known positive constant. 

Let denote the difference of the risks of estimate and that of ([2]) as 

A{9) ■.= R{9*,9)-R{9ml,9). 
We will need also the following constant 

2p/2-ir (I) d 

where fj, = d/ y/aF, 

^ /-^ exp(-7^^^ and d = sup{|l^|| : ^ G O}. 

Jo + r 

Theorem 2.1. Let the noise in (j4]) have a conditionally Gaussian distribu- 
tionAfp{0, 'DlQ)) and its covariance matrixT>{Q) satisfy conditions (Ci), (C2) 
with some compact set G C W. Then the estimator (E]) with c= {p— l)A*7p 
dominates the MLE 9ml for any p >2, i.e. 

supA(e)<-[(p-l)A,7j'. 
Proof. First we will establish the lower bound for the random variable 

Lemma 2.2. Under the conditions of Theorem 2.1 

inf Ee-^ > 7p. 
The proof of lemma is given in the Appendix. 

In order to obtain the upper bound for A(0) we will adjust the argument 
in the proof of Stein's lemma [9] to the model (j4]) with a random covariance 
matrix. 

We consider the risks of MLE and of (|5]) 

r{9ml. e) = E.ii^ML - = E,(E||^ML - e\\''\g) = Y,etrV{g)- 

R{9\9) = R{9ML,9) + Y.e[m9{y) " l)'rf|6^)] 

p 

+2^E4E((^7(r)-i)r,(r,-e,)|^)], 
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where g(Y) = l-c/||F||. 

Denoting f(Y) = {g{Y) — l)Yj and applying the conditional density of 
distribution of a vector Y with respect to a-algebra Q 



"-^^"'^^ ^2.)WVdetD(g) ^^^ 



{x - eyv-\g){x - e) 



one gets 



Ij:=E{f{Y){Y,-9,)\g)= [ f{x){x-9,)pY{x\g)dx, j = l,p. 

J UP 

Making the change of variable u = T)~'^^'^(Q)(x — 9) and assuming f(u) 
f{V^/\g)u + 9), one finds that 



\u\ 

' (2vr) 



where {A)ij denotes the (z,j)-th element of matrix A. These quantities can 
be written as 



1=1 k=i 

Thus, the risk for an estimator ([5]) takes the form 



R{9\9) = R{9ML,9) + ^e{{g{Y) - l)lrf ) 

\j=l 1=1 k=l 

Therefore one has 

R{9\9) = R{9ML.9) + ^eW{Y), 

where 

z'V{g)z ^.^ 1 



W{z) =c' + 2c ,,^,,3^ - 2trV{G)c 



\z\ 



This implies that 

A{9) = BeW{Y). 
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Since z' Az < Xmax{A)\\z\\'^, one comes to the inequality 



A{e) <c^ - 2cE, 
From here it follows that 



\Y\ 



i=2 



\Y\ 



Taking into account the condition (Ci) and the Lemma 12.21 one has 

A{e) <c^ - 2{p - l)A,7pC =: 0(c). 

Minimizing the function (f){c) with respect to c, we come to the desired 
result 

A(^)<-[(p-l)A,7p]'. 

Hence Theorem 12.11 

Corollary 2.3. Let in (j4]) the noise ^ ~ A/^(0, D) with the positive definite 
non random covariance matrix D > and Xmin{D) > A* > 0. Then the 
estimator with c = {p — l)A*7p dominates the MLE for any p > 2 and 
compact set C W, i.e. 

supA(^^)<-[(p-l)A,7p]'. 

eee 



Remark 2.1. Note that if D = a'^Ip then 

snpA{9)<-[ip-l)a\]'. 

Corollary 2.4. If i ^ Mpifd, Ip) and 6 = in model 
estimate ([5]) is given by the formula 



□ 



then the risk of 



R{0,9*)=p 



(p-l)r((p-l)/2) 

V2r{p/2) 



(6) 
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Figure 1: Risk of 6* at 6 = 0. 

By applying the Stirling's formula for the Gamma function 

T{x) = y2^a;^~^/2 exp(-a;) (1 + o(l)) 

one can check that — )■ 0.5 as p — t- oo. The behavior of the risk (jH]) for small 
values of p is shown in Fig.l. It will be observed that in this case the risk of 
the James-Stein estimate remains constant for all p > 3, i.e. 

Ri0jjs) = 2 

and the risk of the MLE 6ml is equal to p and tends to infinity as p — oo. 
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3 Improved estimation in a non-Gaussian Ornstein— 
Uhlenbeck— Levy regression model 

In this section we apply the proposed estimate ([5]) to a non-Gaussian con- 
tinuous time regression model Let observations {yt)o<t<n obey the equation 

p 

dyt = ej(f)j{t)dt + d^t, 0<t<n. (7) 
i=i 

Here a vector 6 = (6'i, 6'p)' of unknown parameters from some compact 
set 6 C W. Assume that (0j(t))j>i is a one-periodic M+ — t- M functions, 
bounded and orthonormal in L2[0,l]. The noise {^t)t>o in ([7]) is a non- 
Gaussian Ornstein-Uhlenbeck process given by stochastic differential equa- 
tion 

dC,t = a^tdt + dut, (8) 

where a < is unknown parameter, {ut)t>o is a Levy process satisfying the 
equation 

Ut = QiWt + Q2Zt. (9) 

Here qi, Q2 are unknown constants, {wt)t>o is a standard Brownian motion, 
{zt)t>o is a compound Poisson process defined as 

Nt 

^t = Y.^,. (10) 

where {Nt)t>o is a Poisson process with unknown intensity A > and (y^)j>i 
is a sequence of i.i.d. Gaussian random variables with parameters (0,1). 

The problem is to estimate the unknown vector parameter 9 on the basis 
of observations {yt)o<t<n- 

Let Q = a{Nt,t > 0} denote the a- algebra generated by the Poisson 
process. 

It will be noted that the model ([7]) is conditionally Gaussian given the a- 
algebra Q. Therefore one can use estimate ([5]) to obtain an improved estimate 
of the unknown vector parameter 6. To this end we have to reduce the initial 
continuous time regression model ([7]) to a discrete time model of the form 

The quality of an estimator 6 will be measured by the quadratic risk 

R{e,e) = Ee\\e-e\\\ 



8 



A commonly used estimator of an unknown vector 9 in model ([7]) is the 
least squares estimate (LSE) 6 = {6i, ...,6^)' with 

in 

= - / (Pjit)dyt, j = 



n 



From here taking into account ([7]), one has 

e = e + e„.an), (11) 

where e„ = n'^^"^ and C(n) is the random vector with coordinates 

Qin) = 4= / U^Ht- 



Note that the vector ({n) has a conditionally Gaussian distribution with a 
zero mean and conditional covariance matrix Vn{G) = cov{({n), C(^)'l^) with 
the elements 

Vi,{n) = E(OHOH I g). 

Thus the initial problem of estimating parameter 6' in (JTj) can be reduced 
to the that of estimating parameter 6 in conditionally Gaussian regression 
model f lTTj) . 

Theorem 3.1. Let the regression model be given by the equations (171) -( fTOl) . 
^1 > 0. Then, for any n >2 and p > I, the estimator of 9 

V ^ll^ll 

dominates the LSE 6: 



sup A(e) < - 



n 



To proved this theorem one can apply Theorem 12.11 and it suffices to check 
conditions (Ci), (C2) on the matrix V^(^). The proof of conditions (Ci) and 
(C2) is given in the Appendix. 
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4 Improved estimation in an autoregression 



In this section we consider the problem of improved estimating the unknown 
mean of a multivariate normal distribution when the dispersion matrix is 
unknown and depends on some nuisance parameters. 

Let in (jlj) the noise C, = (^i, ...,(,0)', be described by a Gaussian autore- 
gression process 

= <fc-i + £k, k = l,p, (12) 
where \a\ < 1, E^o = and independent Gaussian (0,1) random 

variables. Assume that the parameter a in f lT2|) is unknown and belongs to 
interval [—a, a], where < a < 1 is known number. 

It is easy to check that the covariance of the noise ^ has the form 



/ 1 

a 



Via) 



l-a2 



a 
1 



J 



Proposition 4.1. Let in be specified by (1121) with a G [—a, a]. Then 
for any p > 1/(1 — a)^ the MLE is dominated by the estimator 

l-(p ^ 



I -a) 



\Y\ 



Y 



and 



supA(^) < 



1 



p 



7'- 



Proof. One has that trV{a) = p/{l — a^). Now we find the estimation of 
the maximal eigenvalue of matrix P(a). By definition 

>^max{V{a)) = sup z'V{a)z 



one has 
z'V(a)z 



5Z 5Z < ^^^^ 
i=i j=i 



p—l p—i 



1 -a2 



1 + 



a^ ZjZ 



i=i j=i 



l-a2 



1 + 2^a-'^2;jZi+j 
j=i i=i 
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By applying the Cauchy-Bunyakovskii inequahty we obtain that 

if °o \ 2 
A_(P(a))<^(^l + 2X:«'J=(^. 

Thus, 

trV{a) - \max{T^{o)) >p - jz r^. 

(1 

Hence, taking into account the Theorem 12.11 we come to assertion of Propo- 
sition. 



5 Conclusions 

In this paper we propose a new type improved estimation procedure. The 
main difference from the well-known James-Stein estimate is that in the dom- 
inator in the corrected term we take the first power of the observation norm 
This allow us to improve estimation with respect to MLE begining with 
any dimension p > 2. Moreover, we apply this procedure to the estimation 
problem for the non-Gaussian Ornstein-Uhlenbeck-Levy regression model. 



6 Appendix 

6.1. Proof of the Lemma [2. 2[ 
Proof. From (JM one has 



J — Efl— — = Efl— — > Efl 



\y\\ \\o+a - d+uw 

Using a repeated conditional expectation and since the random vector ^ is 
distributed conditionally normal with a zero mean, then 



{27r)p/^^deW{g) Jrp d + \\x 

Making the change of variable u = T>{Q)~^^'^x and applying the estimation 
u'V{g)u < \„,axiV{Q))\\uf we find 

-P'-IIHI72) .... 



Further making the spherical changes of the variables yields 
J > .V ,.E. r '■'-exp(-rV2) ,^ 



From here by Jensen and Cauchy-Bunyakovskii inequalities and by the 
condition (C2) we obtain 

r r^-^exp(-rV2) 
This leads to the assertion of Lemma I2.2[ 

□ 

6.2. The proof of conditions (Ci) and (C2) on the matrix Vn{Q). 
The elements of matrix Vn{Q) can be written as [S] 

Vij{n) = — (j)i{t)(f)j{t)dt 
^ Jo 

+ (0.(t)L^^.(t-Tz,TO + 0,(t)L^,(t-T;,TO)xm<t)cit, (13) 

where 

£g{t) = a exp(a(t — s))5'(s)(l + exp{2as))ds, 
Jo 

Lg{x,y) = aexp{ax) ^g{y) + a J exp{as)g{s + y)ds^ 

and (Ti)i>i are the jump times of the Poisson process {Nt)t>o, i-e. 

Ti = mi{t >0: Nt = l}. 

Lemma 6.1. Let {C,t)t>o be defined by with a < 0. Then a matrix Vn{G) = 
(f jj(0))i<jj<p with elements defined by fll3p . satisfy the following inequality 
a.s. 

inf inf z'Vn{G)z > gf. 

n>l \\z\\ = l 
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Proof. Notice that by fll3p one can the matrix Vn{G) present as 
where F„ is non random matrix with elements 

and Bn{Q) is a random matrix with elements 

hj{n) = glY^[(j)i{Ti)(f)j{Ti)x{Ti<n) 
i>i 

+ / {(l),it)L^^it-Ti,Ti) + (l),it)L^Xt-Ti,Ti))xm<t)dt]. 
Jo 

This implies that 

z'Vn{g)z = q\z'z + z'FnZ + z' B.a{g)z > q\z!z, 

therefore 

\^^^:^V^{Q)z>q\ 
and we come to the assertion of Lemma 16.11 

□ 

Lemma 6.2. Let {C,t)t>o be defined by (IH]) with a < 0. Then a maximal 
eigenvalue of the matrix Vn{Q) = (f^j(^^))l<^,j<p with elements defined by 
( fT3l) . satisfy the following inequality 

SUpSUpE0Ar„ax(K(^)) < MpQ* , 

n>i See 

where g* = gf + Xgl and M > 0. 
Proof. One has 

p * p 

3=1 ^ i=i 
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where 

nn 

Tj{n) = / 0|(t)dt 
Jo 

+ a (pjit) / exp(a(t — u))(l)j{u){l + exp{2au))dudt. 
Jo Jo 

Since the {4>j)i<j<p is a one-periodic orthonormal functions therefore the first 
integral is equal to n and in view of the inequality max < K for any 

a < 

pn ft 

a / (j)j{t) / exp{a{t — u))(l)j{u){l + exp{2au))dudt 
Jo Jo 

< 2K^\a\ / / exp(a(t - u))dudt < 2K^n. 
Jo Jo 

From here denoting M = 1 + 2K^ we obtain that 

Eg A 

max 

Hence Lemma [6.21 

□ 

Thus the matrix Vn{Q) is positive definite and satisfies for any compact 
set G C MP, the conditions (Ci) and (C2) with A* = gf and a* = Mpg^. 
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